Lower Memory Oblivious (Tensor) Subspace Embeddings with Fewer Random Bits: Modewise Methods for Least Squares

In this paper new general modewise Johnson--Lindenstrauss (JL) subspace embeddings are proposed that can be both generated much faster and stored more easily than traditional JL when working with extremely large vectors and/or tensors. Corresponding embedding results then proven for two different types of low-dimensional (tensor) subspaces. The first these produces improved space complexity bounds rank-$r$ tensors whose CP decompositions contained in the span a fixed (but unknown) set $r$ rank-$1$ basis vector setting result yields very near-optimal oblivious constructions require fewer random bits to generate standard subspaces $\mathbb{C}^N$ spanned by special Kronecker structure. second herein provides fast arbitrary $r$-dimensional $\mathcal{S} \subset \mathbb{C}^N$ which also (and so easier store, i.e., less space) methods order achieve small $\epsilon$-distortions. These work (i) effectively folding any given $\mathcal{S}$ into (not necessarily low-rank) tensor, (ii) resulting tensor $\mathbb{C}^m$ $m \leq C r \log^c(N) / \epsilon^2$. Applications related compression compressed least squares solution considered, including those used fitting low-rank decompositions, shown well numerically settings.